5.9 μs
br

3.7 ms

Welcome to 18.S191 – Fall 2020!

Introduction to Computational Thinking for Real-World Problems

10.6 μs

78.0 ns
416 μs

121 ns
Alan Edelman, David P. Sanders, Grant Sanderson, James Schloss
12.3 μs
& Philip the Corgi
8.3 μs

Class outline

Data and computation

  • Module 1: Analyzing images

  • Module 2: Particles and ray tracing

  • Module 3: Epidemic spread

  • Module 4: Climate change

190 μs

Tools

10.7 μs

Module 1: Images

26.3 μs
8
100 ns

Data takes many forms

  • Time series:

    • Number of infections per day

    • Stock price each minute

    • A piece for violin broadcast over the radio


  • Video:

    • The view from a window of a self-driving car

    • A hurricane monitoring station


  • Images:

    • Diseased versus healthy tissue in a scan

    • Deep space via the Hubble telescope

    • Can your social media account recognise your friends?

4.8 ms

Capture your own image!

14.6 μs
104 ns
Enable webcam
40.7 ms

16.7 μs

UndefVarError: grant not defined

  1. top-level scope@Local: 1
---
grant

MethodError: no method matching getindex(::Missing, ::String)

  1. process_raw_camera_data(::Missing)@Other: 13
  2. top-level scope@Local: 1[inlined]
---

What is an image?

8.5 μs

Albrecht Dürer:

5.5 μs

6.8 μs

An image is:

  • A 2D representation of a 3D world

  • An approximation

42.9 μs

What is an image, though?

  • A grid of coloured squares called pixels

  • A colour for each pair (i,j) of indices

  • A discretization

2.8 ms

How can we store an image in the computer?

  • Is it a 1D array (Vector)?

  • A 2D array (Matrix)?

  • A 3D array (tensor)?

39.5 μs

If in doubt: Ask Julia!

  • Let's use the Images.jl package to load an image and see what we get

11.7 μs
80.0 ms
16.2 s
url
"https://i.imgur.com/VGPeJ6s.jpg"
73.0 ns
philip_file
"philip.jpg"
283 ms
philip
3.7 s
102 ns
Array{RGB{Normed{UInt8,8}},2}
84.0 ns
10.5 ms

5.6 μs
56.0 ns

10.7 μs
Array{RGB{Normed{UInt8,8}},2}
81.0 ns

  • According to Julia / Pluto, the variable philip is an image

  • Julia always returns output

  • The output can be displayed in a "rich" way


  • Arthur C. Clarke:

Any sufficiently advanced technology is indistinguishable from magic.

42.5 μs
253 ns
103 ns
1.6 ms

How big is Philip?

  • He's pretty big:

8.7 μs
246 ns
  • Which number is which?

17.8 μs
67.0 ns

So, what is an image?

25.4 μs
Array{RGB{Normed{UInt8,8}},2}
107 ns
  • It's an Array

  • The 2 means that it has 2 dimensions (a matrix)


  • RGBX{Normed{UInt8,8}} is the type of object stored in the array

  • A Julia object representing a colour

  • RGB = Red, Green, Blue

46.4 μs

Getting pieces of an image

55.7 μs
29.3 ms
254 ns
295 ns

Manipulating matrices

  • An image is just a matrix, so we can manipulate matrices to manipulate the image

17.8 μs
20.9 ms

7.0 μs
1.9 s

Manipulating an image

  • How can we get inside the image and change it?

  • There are two possibilities:

    • Modify (mutate) numbers inside the array – useful to change a small piece

    • Create a new copy of the array – useful to alter everything together

544 μs

Painting a piece of an image

  • Let's paint a corner red

  • We'll copy the image first so we don't destroy the original

21.0 μs
new_phil
27.4 ms

11.4 μs
red
80.1 ms
109 μs

Note that for loops do not return anything (or, rather, they return nothing)

9.3 μs
71.0 ns

Element-wise operations: "Broadcasting"

  • Julia provides powerful technology for operating element by element: broadcasting

  • Adding "." applies an operation element by element

12.8 μs
157 ms

5.7 μs
109 ns

Modifying the whole image at once

  • We can use the same trick to modify the whole image at once

  • Let's redify the image

  • We define a function that turns a colour into just its red component

22.7 μs
redify (generic function with 1 method)
25.2 μs
53.5 ms

17.3 μs
218 ms
193 ms

Transforming an image

  • The main goal of this week will be to transfrom images in more interesting ways

  • First let's decimate poor Phil

16.8 μs
poor_phil
635 μs

6.1 μs

Experiments come alive with interaction

  • We start to get a feel for things when we can experiment!

11.6 μs
17.6 s

7.0 μs
1
42.5 ms
14.7 ms

Summary

  • Images are readily-accessible data about the world

  • We want to process them to extract information

  • Relatively simple mathematical operations can transform images in useful ways

12.5 μs
1.9 ms
expand (generic function with 2 methods)
48.9 μs
extract_red (generic function with 1 method)
24.5 μs
decimate (generic function with 2 methods)
55.8 μs

Appendix

14.1 μs

Package environment

11.8 μs

Camera input

13.3 μs
camera_input (generic function with 1 method)
92.9 μs
process_raw_camera_data (generic function with 1 method)
59.9 μs